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Background / Context: 

Description of prior research and its intellectual context. 

Improvements in students’ learning achievement have lagged behind in low-and middle-income 
countries despite significant progress in school enrollment numbers. Large-scale early grade 
reading assessments (e.g., Annual Status of Education Report [ASER], 2013; EdData II, n.d.) 
have shown low reading rates and worryingly high “zero” scores in reading assessments from 
across the world, including Latin-America. Numerous initiatives are underway globally to try to 
improve children’s literacy development. Although several studies focus on the impact of these 
initiatives in low-and middle-income countries in Latin-America, to the best of our knowledge , 
there exists no systematic synthesis of the evidence on what works to improve early grade reading 
outcomes in the Latin- American and Caribbean region. 

It is crucial, however, to assess what works to improve reading outcomes in the context of Latin- 
America and the Caribbean and why these programs have these effects on reading outcomes. 
Although reading outcomes are improving in the LAC region, there exists major heterogeneity in 
learning outcomes across countries both in terms of levels and trends (UNESCO, 2015). Thus, 
assessing what works and why in improving reading outcomes may allow governments and other 
stakeholders in Latin-America to replicate and scale programs that are successful in improving 
reading outcomes in the LAC region. 

Purpose / Objective / Research Question / Focus of Study: 

Description of the focus of the research. 

This systematic review synthesizes the existing literature on what works to improve early grade 
reading outcomes in the Latin- American and Caribbean (LAC) region. The review addresses the 
following research questions: 

1. What is the impact of reading programs, practices, policies and products aimed at 
improving the reading skills for children from birth through grade 3 on reading outcomes 
in the LAC region? 

2. What are the gaps in the evidence base on early grade reading in the LAC region? 

We will synthesize the evidence from experimental and quasi-experimental studies to address 
these research questions. In addition, we will synthesize the qualitative evidence associated with 
reading outcomes in the LAC region and evidence linked to reading outcomes but not associated 
with a specific intervention. 

The overarching goals of this review is to (1) increase the availability of information for 
evidence-based decision making for international agencies, NGOs, and government policy 
makers who select programming for children, and (2) identify evidence gaps regarding the 
effectiveness of interventions currently in use. 

Setting: 

Description of the research location. 
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We include evidence from or on the LAC region with an emphasis on the reading outcomes of 
boys or girls ages birth through grade 3 - regardless of the age of the child. We will include all 
research that meets these inclusion criteria. Then we will make a distinction between quantitative 
studies that focus on the effectiveness of interventions, non-intervention based quantitative 
studies, qualitative studies that focus on specific interventions, and non-intervention related 
qualitative studies. 

Population / Participants / Subjects: 

Description of the participants in the study: who, how many, key features, or characteristics. 

We will only include studies that focus on early grade reading. For this purpose we will use a 
definition of early grade reading that is used by USAID. Early grade reading is defined by 
USAID as pertaining to grades 1-3 of primary schooling. Our systematic review will focus on 
this population of students but will also broaden the definition to include children from birth, as 
there is a large evidence base on the importance of developing early language skills, exposure to 
print, and pre-reading activities for improving later reading success. Furthermore, we will 
include any children from birth through grade 3 regardless of age as we are aware that in many 
countries children up to 1 1 years old may still be in the third grade due to late entry and grade 
retention policies. 

Intervention / Program / Practice: 

Description of the intervention, program, or practice, including details of administration and duration. 

We will include evidence from all programs that focus on improving reading outcomes in the 
LAC region. For this purpose we will liberally apply the inclusion criteria to ensure that all 
relevant literature is included and nothing is excluded without thorough evaluation. Following an 
assessment of relevance we will conduct a risk of bias assessment as described under the 
research design. 

Research Design: 

Description of the research design. 

We will conduct a mixed-methods systematic review in order to benefit from evidence from 
quantitative and qualitative studies that focus on interventions that aim to improve reading 
outcomes as well as non-intervention research that focuses on reading outcomes in the LAC 
region. 

Quantitative Studies with a Focus on Effectiveness 

After collecting all the quantitative studies with an emphasis on the effectiveness of programs for 
review, we will code and critically appraise the quality of the literature using a risk of bias 
assessment. Coding and critically appraising the literature are necessary because findings from 
low-quality evaluation studies may be biased. We will determine the rigor of the quantitative 
studies using an adaptation of a tool developed by Hombrados & Waddington (2012). This tool 
specifically focuses on the likelihood of 1) selection bias, 2) bias from spillovers, 3) outcome and 
analysis reporting bias, and 4) other biases such as relying on recall data or not accounting for 
clustering in the standard errors. We will use the tool to determine whether studies should be 
considered high, medium, or low risk of bias for each of the four categories. 
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We will calculate effect sizes for all quantitative research studies that meet the criteria for 
inclusion in the meta- analysis. We will use the standardized mean difference to make effect sizes 
on categorical variables comparable across studies. The standardized mean difference is the most 
common way to estimate the effect size of an intervention. This measure can be used for 
categorical variables (e.g., learning outcomes) and divides the mean difference between two 
groups by the standard deviation of this difference to make effect sizes comparable across studies 
as in equation 1 (Lipsey & Wilson, 2001): 

Where possible, we will combine quantitative study results using meta-analysis. We will 
examine the heterogeneity of the effect sizes for each outcome across studies, and use meta- 
regression to model the variation in effect size. For the meta-analysis we will only include 
studies with an emphasis on interventions that use one of the following designs: 1) experimental 
designs using random assignment to the intervention and 2) quasi-experimental designs with 
nonrandom assignment (such as regression discontinuity designs, “natural experiments,” and 
studies in which participants self-select into the program). To be included, the studies need to: 1) 
collect data at baseline and endline (longitudinal) and/or cross-sectional (endline) data from 
treatment and comparison groups; and 2) use statistical matching, difference-in differences 
estimation, instrumental variables regression, multivariate cross-sectional regression analysis; or 
other forms of multivariate analysis (such as the Heckman selection model or multivariate OLS 
regression analysis) that are able to correct for selection bias under specific circumstances. We 
will include studies with data collected at the individual and/or group level. For studies that 
utilized interrupted time series, at least three data points needed to be collected before and after 
the intervention for the study to be included. Eligible comparison conditions will be no 
intervention, pipeline, or “business as usual.” We will also include studies for which we are able 
to calculate the effect size and associated standard error. If the necessary data to calculate effect 
sizes are not available in the included studies, we will contact the authors of the studies. 

We will start our analysis with separate meta-analyses of randomized controlled trials and quasi- 
experimental evaluations for determining the effects of interventions to improve reading 
outcomes. Then we will follow an iterative approach to determine the potential bias from pooling 
randomized controlled trials and quasi-experimental evaluations and studies with low, medium, 
and high risk of bias for each of the types of bias we assessed in our risk of bias assessment. We 
will use random-effects meta-analysis because of potential differences in the context and 
interventions that focus on reading outcomes. 

Qualitative Studies with a Focus on the Intervention 

For the qualitative studies we will use the 9-item Critical Appraisal Skills Program Qualitative 
Research Checklist (CASP, 2013) to determine the risk of bias of the included studies. The use 
of this tool will allow us to make judgments on the adequacy of stated aims, the data collection 
methods, the analysis, the ethical considerations and the conclusions drawn. High-quality 
qualitative studies are contributory, defensible in design, rigorous in conduct, and credible in 
claim (Spencer et al., 2003; Snilstveit et al., 2012). We will use a narrative synthesis approach to 
synthesize the findings of the qualitative studies. 
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Other Studies with a Focus on Reading Outcomes 

We will critically appraise the quality of other studies with a focus on reading outcomes using a 
more theoretical approach. This theoretical approach Simple View of Reading (Gough & 
Tunmer, 1986), which states that, in essence, reading success is contingent upon two main 
abilities: language ability and decoding abilities, and the sub-skills of each of these. We will 
determine the extent to which studies that are not related to interventions take this theoretical 
approach into consideration. Then we will narratively synthesize the evidence of the other 
studies with a focus on reading outcomes. 

Data Collection and Analysis: 

Description of the methods for collecting and analyzing data. 

To conduct the systematic review, we will use four primary methods of searching for early grade 
reading evidence in the LAC region: 

1. Internet searches of pre-defined databases, journals and international development 
organizations 

2. Development focused databases/websites 

3. LAC Region Databases and Websites 

4. A review of bibliographies or references of the included studies 

For this review we will use several computational approaches that have been effective in 
identifying documents relevant to the selected population and topic. Such algorithms include 
topic modeling, information retrieval and clustering, and search term strategies widely used in 
library science and biblio metrics. The computational approach will allow us to identify 
potentially relevant studies, which will subsequently be further analyzed by reading and 
methodological experts. 

We will also identify grey literature by relying on Google scholar, and surveys with experts in 
the field. Including grey literature in the systematic review will be crucial in order to minimize 
the risk of publication bias. 

Findings / Results: 

Description of the main findings with specific details. 

After our first search we identified 7080 studies that may be considered relevant for this review. 
We then applied the computational information retrieval and clustering approach to get more 
relevant results, which returned 1,203 abstracts for review by reading and methodological 
experts. We will now analyze the relevance and methodological quality of the studies. 

Conclusions: 

Description of conclusions, recommendations, and limitations based on findings. 

We found a wide variety of literature that can be considered relevant for the context of early 
grade reading in Latin- America and the Caribbean. During the conference we will discuss the 
results of the narrative review and the meta-analysis. At this moment these results are not yet 
known. 
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